Performing an action associated with a motion based input

ABSTRACT

A method implemented by a computing device having a memory and a motion sensor detection sensor is disclosed. The memory stores an input model associated with a gesture input. The method includes detecting a motion of the computing device; matching the detected motion with the input model; and determining a confidence level associated with the match. If the confidence level is above a pre-determined threshold, an action associated with the gesture input is performed automatically by the processor; and if the confidence level is below the pre-determined threshold, the action is performed by the processor responsive to receiving a positive confirmation of the gesture input.

RELATED APPLICATION DATA

The present application is a continuation of U.S. patent applicationSer. No. 13/903,206, May 28, 2013, now U.S. Pat. No. 10,078,372 B2,issued the content of which is incorporated herein by reference.

TECHNICAL FIELD

The present disclosure relates to a method and apparatus for performingan action associated with a motion based input.

BACKGROUND

Human-computer interaction is enhanced when the methods of detectinguser input feel natural to the user. This has prompted an increase incomputing devices having a touch sensitive display suited for receivinguser inputs and further suited to act as a primary human-computerinteraction interface. However, other human-computer interactioninterfaces are possible.

BRIEF DESCRIPTION OF THE DRAWINGS

Reference will now be made, by way of example, to the accompanyingdrawings which show example embodiments of the present application, andin which:

FIG. 1 is a block diagram illustrating a mobile computing device havinga motion detection subsystem and suitable for detecting a motion basedinput;

FIG. 2 is a perspective view of the mobile computing device of FIG. 1with a three-axis motion detection subsystem mounted therein inaccordance with one example embodiment of the present disclosure;

FIG. 3 is a block diagram of a circuit of a motion detection subsystemsuitable for use with the mobile computing device of FIG. 1 inaccordance with one example embodiment of the present disclosure;

FIGS. 4A to 4C are schematic diagrams illustrating the assignment ofpitch and roll vectors of a three-axis accelerometer in accordance withone example embodiment of the present disclosure;

FIG. 5 is a flow diagram illustrating a method for performing an actionassociated with a motion based input;

FIG. 6A is a plot illustrating a first set of sample data from a motiondetection subsystem, of the mobile computing device of FIG. 1, comparedwith a first example input model; and

FIG. 6B is a plot illustrating the first set of sample data from themotion detection subsystem, of the mobile computing device of FIG. 1,compared with a second example input model; and

FIGS. 7A and 7B are plots illustrating sets of sample data from a motiondetection subsystem, of the mobile computing device of FIG. 1, comparedwith the first example input model.

Similar reference numerals may have been used in different figures todenote similar components.

DESCRIPTION OF EXAMPLE EMBODIMENTS

An input device that is suited for detecting motion and furtherinterpreting the detected motion as a motion based input may in somescenarios provide a human-computer interaction interface that feelsnatural to a user. For example, making a phone call using a mobile phonemay involve selecting a phone number then moving the phone to one's ear.The motion associated with moving a mobile phone to one's ear can bedetected using a motion detection subsystem of the mobile phone, whichmay then be characterized as a detected motion by a processor. Thecharacterized motion can then be compared by the processor with apreviously stored input model to determine if a match between thedetected motion and the stored input model exists. When a match isdetermined to exist, an action associated with the input model may thenbe performed, for example the action may be to place a call.

A continuous stream of data input from the motion detection subsystemcan be provided to the processor. The continuous stream of data input isanalyzed in real-time by the processor; thus if at any moment theprocessor determines that a match is found, the associated action willbe performed. However, the matching algorithm implemented may output awrong determination. For example, when the mobile phone is removed froma purse, the detected device motion may match the input model and thuscause the action associated with the input model to be performedunwanted. The unwanted action may be problematic for users in many ways,depending on the action performed and when it is performed.

In accordance with one aspect of the present disclosure, there isprovided a method implemented by a computing device for performing anaction associated with a motion based input. The method may beimplemented by a computing device comprising a processor, and a memory,such as a non-transitory computer readable medium, and motion detectionsensor coupled to the processor. The memory stores an input modelassociated with a gesture input. The method comprises detecting a motionof the computing device; matching the detected motion with the inputmodel; determining a confidence level associated with the match; if theconfidence level is above a pre-determined threshold, performing anaction associated with the gesture input; and if the confidence level isbelow the pre-determined threshold, performing the action responsive toreceiving a positive confirmation of the gesture input. The memory hasstored thereon executable instructions that, when executed by theprocessor of the computing device, configure the computing device toperform the method.

In some embodiments, the method further comprises determining the actionassociated with the input model based on a temporal context, wherein thetemporal context is a state of an application when the motion isdetected.

In some embodiments, determining the confidence level further comprisescomparing the detected motion with the input model stored in memory. Thedetected motion may also be stored in memory as an additional inputmodel after receiving the positive or negative confirmation of thegesture input. The input model may also be adjusted in response toreceiving the positive confirmation or a negative confirmation of thegesture input.

In some embodiments, the method further comprises prior to detecting themotion, receiving a selection in adjusting the input model in responseto receiving a negative confirmation of the gesture input indicatingselection of a message; and wherein performing the action comprisessending a reply to the message.

In some embodiments, the method further comprises prior to detecting themotion, receiving a selection indicating selection of a contact entry,the contact entry having a plurality of phone numbers stored in memory.In some embodiments, performing the action further comprises promptingthe user to select a phone number of the plurality of phone numbers fordialing, receiving a voice input selecting the phone number and dialingthe phone number.

In some embodiments, receiving a confirmation comprises prompting auser, for example via a voice prompt, to confirm the gesture input, andreceiving an input, for example a voice input, indicating the positiveconfirmation.

In accordance with another aspect of the present disclosure, there isprovided a method implemented by a computing device comprising aprocessor, and a memory, motion detection sensor and proximity sensorcoupled to the processor, the method comprising: measuring, using themotion detection sensor, motion data representing a detected motion ofthe computing device at regular intervals; measuring, using theproximity sensor, proximity data representing a proximity of thecomputing device to an object at regular intervals over a duration ofthe detected motion; matching the motion data and proximity data with aninput model associated with a gesture using a plurality of criteriadefined by the input model, the plurality of criteria comprising apattern of proximity data representing changes in the proximity of thecomputing device to the object during the detected motion and a patternof motion data representing the detected motion; determining aconfidence level associated with the match based on how closely themotion data and proximity data match the plurality of criteria in theinput model; in response to determining the confidence level is above athreshold, performing an action associated with the gesture; and inresponse to determining the confidence level is below the threshold,prompting for designated confirmatory input, wherein the designatedconfirmatory input is different from the gesture, and performing theaction associated with the gesture in response to receiving designatedconfirmatory input indicating a positive confirmation to perform theaction.

In some embodiments, the method further comprises: determining theaction associated with the gesture based on a state of an applicationwhen the motion data was measured.

In some embodiments, the method further comprises: storing the motiondata in association with the proximity data in the memory as anadditional input model in response to receiving the designatedconfirmatory input indicating a positive confirmation to perform theaction or designated confirmatory input indicating a negativeconfirmation to not perform the action.

In some embodiments, the method further comprises: adjusting the inputmodel in response to receiving the designated confirmatory inputindicating a positive confirmation to perform the action or designatedconfirmatory input indicating a negative confirmation to not perform theaction.

In some embodiments, the method further comprises: prior to measuringthe motion data and proximity data, receiving a selection indicatingselection of a contact entry, the contact entry having a plurality ofphone numbers stored in memory; and wherein performing the actioncomprises: prompting a user to select a phone number of the plurality ofphone numbers for dialing; receiving a voice input selecting the phonenumber; and dialing the phone number.

In some embodiments, prompting comprises announcing, via a speaker ofthe computing device, a voice prompt, wherein designated confirmatoryinput indicating a positive confirmation to perform the action ordesignated confirmatory input indicating a negative confirmation to notperform the action comprising a voice input is received via a microphoneof the computing device.

In some embodiments, prompting comprises displaying a message on adisplay of the computing device, and wherein designated confirmatoryinput indicating a positive confirmation to perform the action ordesignated confirmatory input indicating a negative confirmation to notperform the action is received via an input device of the computingdevice.

In some embodiments, the input device is a hardware keyboard. In otherembodiments, the input device is a touch-sensitive display.

In some embodiments, the method further comprises: selecting an inputmodel, from a plurality of input models, based on a state of anapplication when the motion data was measured.

In some embodiments, the pattern of motion data is defined by adisplacement of the detected motion correlated to a distance thecomputing device is moved when the gesture input is performed, a speedof the detected motion correlating a speed at which the computing deviceis moved when the gesture input is performed, a direction of thedisplaced distance, and a frequency response of the detected motion.

In some embodiments, each criterion in the plurality of criteria of theinput model is compared individually against the motion data orproximity data.

In some embodiments, a weighing function is applied to each criterion inthe plurality of criteria of the input model based on the significanceof the criterion.

In accordance with yet a further aspect of the present disclosure, thereis provided a computing device, comprising a processor and a memory,wherein the memory has stored thereon executable instructions that, whenexecuted by the processor of the computing device, configure thecomputing device to perform one or more of the methods described aboveand herein.

In accordance with yet a further aspect of the present disclosure, thereis provided a computing device comprising: a memory for storinginstructions and for storing an input model associated with a gesture; amotion detection sensor; a proximity sensor; a processor coupled to thememory, the proximity sensor and the motion detection sensor, whereinthe processor is configured to: measure, using the motion detectionsensor, motion data representing a detected motion of the computingdevice at regular intervals; measure, using the proximity sensor,proximity data representing a proximity of the computing device to anobject at regular intervals over a duration of the detected motion;match the motion data and proximity data with an input model associatedwith a gesture using a plurality of criteria defined by the input model,the plurality of criteria comprising a pattern of proximity datarepresenting changes in the proximity of the computing device to theobject during the detected motion and a pattern of motion datarepresenting the detected motion; determine a confidence levelassociated with the match based on how closely the motion data andproximity data match the plurality of criteria in the input model; inresponse to determining the confidence level is above a threshold,perform an action associated with the gesture; and in response todetermining the confidence level is below the threshold, prompt fordesignated confirmatory input, wherein the designated confirmatory inputis different from the gesture, and perform the action associated withthe gesture in response to receiving designated confirmatory inputindicating a positive confirmation to perform the action.

In some embodiments, the processor is further configured to: determinethe action associated with the gesture based on a state of anapplication when the motion data was measured.

In some embodiments, the processor is further configured to: store themotion data in association with the proximity data in memory as anadditional input model in response to receiving the designatedconfirmatory input indicating a positive confirmation to perform theaction or designated confirmatory input indicating a negativeconfirmation to not perform the action.

In some embodiments, the processor is further configured to: adjust theinput model in response to receiving the designated confirmatory inputindicating a positive confirmation to perform the action or designatedconfirmatory input indicating a negative confirmation to not perform theaction.

In some embodiments, the processor is further configured to: receive,prior to measuring the motion data and proximity data, a selectionindicating selection of a contact entry, the contact entry having aplurality of phone numbers stored in memory; and when performing theaction: prompt a user to select a phone number of the plurality of phonenumbers for dialing; receive a voice input selecting the phone number;and dial the phone number.

In some embodiments, the processor is further configured to prompt byannouncing, via a speaker of the computing device, a voice prompt,wherein designated confirmatory input indicating a positive confirmationto perform the action or designated confirmatory input indicating anegative confirmation to not perform the action comprising a voice inputis received via a microphone of the computing device.

In accordance with yet a further aspect of the present disclosure, thereis provided a non-transitory machine readable medium having tangiblystored thereon executable instructions that, when executed by aprocessor of a computing device, cause the computing device to performone or more of the methods described above and herein.

Reference is made to FIG. 1 which illustrates an example mobilecomputing device 130 (referred to hereinafter as merely mobile device130 for convenience) suitable for detecting a motion based input.Examples of the mobile device 130 include, but are not limited to, amobile phone, smartphone or superphone, tablet computer, notebookcomputer (also known as a laptop, netbook or ultrabook computerdepending on the device capabilities), wireless organizer, personaldigital assistant (PDA), electronic gaming device, and special purposedigital camera. In some embodiments, the mobile computing device 130 isan accessory device, receiving instructions from a second computingdevice (not shown). Examples of an accessory device may include, withoutlimitation, any motion sensing input device.

The mobile device 130 includes a rigid case (not shown) housing theelectronic components of the mobile device 130. The electroniccomponents of the mobile device 130 are mounted on a printed circuitboard (not shown). The mobile device 130 includes a processor 102 whichcontrols the overall operation of the mobile device 130. Communicationfunctions, including data and voice communication, are performed througha communication interface 104. The communication interface 104 receivesmessages from and sends messages via the communication network 150. Thecommunication interface 104 typically includes a WWAN interface forcommunication over cellular networks and a WLAN interface forcommunication over Wi-Fi networks.

The processor 102 interacts with other components, such as one or moreinput devices 106, RAM 108, ROM 110, a display 112, persistent(non-volatile) memory 120 which may be flash memory or any othersuitable form of memory, auxiliary I/O subsystems 150, data port 152such as serial data port (e.g., Universal Serial Bus (USB) data port),camera 154 such as video and/or still camera, speaker 156, microphone158, a motion-detection subsystem 170, a global positioning system (GPS)device 160 which enables the processor 102 to determine GPS coordinates(i.e., location) of the mobile device 130 at any appropriate time,proximity sensor 162 which enables the processor 102 to determine thedistance between the mobile device 130 and an object at any appropriatetime, and other device subsystems generally designated as 164. Thecomponents of the mobile device 130 are coupled via a communications bus(not shown) which provides a communication path between the variouscomponents.

The display 112 may be provided as part of a touchscreen which providesan input device 106. The display 112 which together with atouch-sensitive overlay (not shown) operably coupled to an electroniccontroller (not shown) comprise the touchscreen. User-interaction withthe GUI is performed through the input devices 106. Information, such astext, characters, symbols, images, icons, and other items are renderedand displayed on the display 112 via the processor 102. The processor102 may interact with the motion-detection subsystem 170 to detectdirection of gravitational forces or gravity-induced reaction forces soas to determine, for example, the orientation of the mobile device 130in order to determine a screen orientation for the GUI.

The input devices 106 may include a keyboard, control buttons (notshown) such as a power toggle (on/off) button, volume buttons, camerabuttons, general purpose or context specific buttons, ‘back’ or ‘home’buttons, phone function buttons, and/or a navigation device. When thedisplay 112 is provided as part of a touchscreen, the various buttons orcontrols may be provided by onscreen user interface elements displayedon the display 112 instead of, or in addition to, physical interfacecomponents. The keyboard may be provided instead of, or in addition to,a touchscreen depending on the embodiment. At least some of the controlbuttons may be multi-purpose buttons rather than special purpose ordedicated buttons.

The mobile device 130 may also includes a memory card interface 130 forreceiving a removable memory card 132 comprising persistent memory, suchas flash memory. A removable memory card 132 can be inserted in orcoupled to the memory card interface 130 for storing and reading data bythe processor 102 including, but not limited to still images andoptionally video images captured the camera 154. Other types of userdata may also be stored on the removable memory card 132. Other types ofremovable digital image storage media, such as magnetic hard drives,magnetic tape, or optical disks, may be used in addition to, or insteadof, the removable memory card 132.

The processor 102 operates under stored program control and executessoftware modules 176 stored in memory, for example, in the persistentmemory 120. The persistent memory 120 also stores data 190 such as aninput model 192 representing a gesture input. As illustrated in FIG. 1,the software modules 176 comprise operating system software 178 andsoftware applications 180. The software applications 180 may include agesture recognition application 181, a contacts application 182, a calllog application 183, an email application 184, a Short Messaging Service(SMS) application 185, a Voice over Internet Protocol (VoIP) application186, a phone calling application 187, a speech application 188, andother applications. The software modules 176 or parts thereof may betemporarily loaded into volatile memory such as the RAM 108. The RAM 108is used for storing runtime data variables and other types of data orinformation, such as data generated in real-time by the motion-detectionsubsystem 170 stored as detected-motion 194. Although specific functionsare described for various types of memory, this is merely one example,and a different assignment of functions to types of memory could also beused.

The communication interface 104 may include a short-range wirelesscommunication subsystem (not shown) which provides a short-rangewireless communication interface. The short-range wireless communicationinterface is typically Bluetooth® interface but may be another type ofshort-range wireless communication interface including, but not limitedto, an IR interface such as an IrDA interface, an IEEE 802.15.3ainterface (also referred to as UWB), Z-Wave interface, ZigBee interfaceor other suitable short-range wireless communication interface.

A received signal, such as a text message, an e-mail message, or webpage download, is processed by the communication subsystem 104 and inputto the processor 102. The processor 102 processes the received signalfor output to the display 112 and/or to the auxiliary I/O subsystem 150.A subscriber may generate data items, for example e-mail messages, whichmay be transmitted over the communication network 150 through thecommunication subsystem 104, for example.

The mobile device 130 also includes a battery 138 as a power source,which is typically one or more rechargeable batteries that may becharged, for example, through charging circuitry coupled to a batteryinterface such as the serial data port 152. The battery 138 provideselectrical power to at least some of the electrical circuitry in themobile device 130, and the battery interface 136 provides a mechanicaland electrical connection for the battery 138. The battery interface 136is coupled to a regulator (not shown) which provides power V+ to thecircuitry of the mobile device 130.

The proximity sensor 162 may comprise a sensor that transmits a field orsignals (such as electromagnetic) to detect the presence of nearbyobjects (i.e. the sensor's target). The maximum distance that theproximity sensor 174 can detect may be predetermined or adjustable.

The mobile device 130 also comprises a motion-detection subsystem 170comprising at least one sensor connected to the processor 102 and whichis controlled by one or a combination of a monitoring circuit andoperating software. In some embodiments, the motion-detection subsystem170 enables to processor 102 to determine if the mobile device 130 is inmotion and the nature of any sensed motion at any appropriate time. Insome embodiments, the motion-detection subsystem 170 enables toprocessor 102 to determine if a person or object is the sensitivityregion of the motion-detection subsystem 170 is moving.

The motion-detection subsystem 170 may comprise one or more sensorsincluding any of accelerometer 172 (such as a three-axis accelerometer),gyroscope 174, magnetometer 176, or other suitable sensor, orcombinations thereof. The motion detection subsystem, or parts thereof,may be combined or shared, for example, within an integrated component.In some embodiments, the motion-detection subsystem 170 may comprise oneor more imaging sensors, including but not limited to imaging sensorssuited to detect infrared light.

The motion-detection subsystem 170 may include one or both of athree-axis accelerometer 172 and a three-axis gyroscope 174, each havingx-, y- and z-sensing axes. As shown in FIG. 2, the sensing axes x-, y-,z- may be aligned with the form factor of the device 130. In someembodiments, the motion-detection subsystem 170 is aligned such that afirst sensing axis x extends longitudinally along the midpoint of thehandheld electronic device 130 between left 226 and right 228 sides ofthe device 130, a second sensing axis y extends laterally along themidpoint of the device 130 between top 222 and bottom ends 224, and athird sensing axis z extends perpendicularly through the x-y planedefined by the x- and y-axes at the intersection (origin) of these axes.It is contemplated that the sensing axes x-, y-, z- may be aligned withdifferent features of the device 130 in other embodiments.

Referring briefly to FIG. 3, motion-detection subsystem 170 inaccordance with one example embodiment of the present disclosure will bedescribed. The circuit 300 comprises a sensor or a group of sensors,including one of or any combination of accelerometer 172, gyroscope 174,magnetometer 176 or other sensors, connected to the interrupt and serialinterface of a controller (MCU) 312. The controller 312 could forexample be implemented by the processor 102 (FIG. 1) of the device 130.The operation of the controller 312 is controlled by software, which maybe stored in internal memory of the controller 312. The operationalsettings of the sensor(s) 310 are controlled by the controller 312 usingcontrol signals sent from the controller 312 to the sensor(s) 310 viathe serial interface. The controller 312 may determine the motion of thedevice in accordance inputs from the sensor(s) 310. The inputs may alsobe sent to the processor 102 (FIG. 1) of the device 130 via its serialinterface where a detected motion is determined by the operating system178, or other software module 176. In other embodiments, a differentdigital configuration could be used, or a suitable analog sensor(s) andcontrol circuit(s) could be used.

As will be appreciated by persons skilled in the art, the accelerometer172 is a sensor which determines acceleration from motion (e.g. movementof the computing device 130) and gravity, which are detected by asensing element and converted into an electrical signal producing acorresponding change in output. The sensor is available in one-, two- orthree-axis configurations. Accelerometers may produce digital or analogoutput signals depending on the type of accelerometer. Generally, twotypes of outputs are available depending on whether an analog or digitalaccelerometer used: (1) an analog output requiring buffering andanalog-to-digital (A/D) conversion; and (2) a digital output which istypically available in an industry standard interface such as an SPI(Serial Peripheral Interface) or I2C (Inter-Integrated Circuit)interface. The output of an accelerometer is typically measured in termsof the gravitational acceleration constant at the Earth's surface,denoted g, which is approximately 9.81 m/s² (32.2 ft/s²) as the standardaverage. The accelerometer may be of almost any type including, but notlimited to, a capacitive, piezoelectric, piezoresistive, or gas-basedaccelerometer. The detection range of accelerometers vary up to thethousands of g's, however for portable electronic devices “low-g”accelerometers may be used. Example low-g accelerometers which may beused are MEMS digital accelerometers from Analog Devices, Inc. (ADI),Freescale Semiconductor, Inc. (Freescale) and STMicroelectronics N.V. ofGeneva, Switzerland.

Referring now to FIGS. 4A to 4C, the assignment of pitch and rollvectors of a three-axis accelerometer in accordance with one exampleembodiment of the present disclosure will be briefly described. Eachsensing axis may be aligned with an axis of the mobile device 130. Asdiscussed above, the x-axis and y-axis are typically aligned with theinput plane of the display 112. The z-axis is perpendicular to thehorizontal plane.

As shown in FIG. 4B, pitch (ϕ) is the angle of the x-axis relative tothe ground. θ is the angle of the z-axis relative to gravity. As shownin FIG. 4C, roll (ρ) is the angle of the y-axis relative to the ground.It will be appreciated that rotation may occur about any combination ofsensing axes. The concepts and methodology described herein can beapplied to any axis orientation and any combination of pitch (ϕ) angle,roll (ρ) angle and θ (the angle of the z-axis relative to gravity).Pitch (ϕ), roll (ρ) and the angle of the z-axis relative to gravity (θ)of a three-axis accelerometer may be calculated using equations (1), (2)and (3):

$\begin{matrix}{\phi = {\arctan\left( \frac{x_{sensor}}{\sqrt{y_{sensor}^{2} + z_{sensor}^{2}}} \right)}} & (1) \\{\rho = {\arctan\left( \frac{y_{sensor}}{\sqrt{x_{sensorl}^{2} + z_{sensor}^{2}}} \right)}} & (2) \\{\theta = {\arctan\left( \frac{\sqrt{x_{sensor}^{2} + y_{sensor}^{2}}}{z_{sensor}} \right)}} & (3)\end{matrix}$

where x_(sensor), y_(sensor) and z_(sensor) are the measurements fromthe x, y and z-axes of the three-axis accelerometer. It will beappreciated that pitch (ϕ), roll (ρ) and the angle of the z-axisrelative to gravity (θ) can also be determined by other means.

As will be appreciated by persons skilled in the art, the gyroscope 174is a sensor which converts angular momentum from motion (e.g. movementof the computing device 130 or a portion thereof due to strike force)into an electrical signal, producing a corresponding change in output,and is available in one-, two- or three-axis configurations. As shown inFIG. 2, the three-axis configuration gyroscopes may have the x-axis andy-axis aligned with the input plane of the display 112 and the z-axisperpendicular to the horizontal plane. However, other configurations arepossible.

Gyroscopes may produce digital or analog output signals depending on thetype of gyroscope. The gyroscope may be of any type including, but notlimited to a MEMS gyroscope, which may produce digital or analog outputsignals. Generally, two types of outputs are available depending onwhether an analog or digital gyroscope is used: (1) an analog outputrequiring buffering and analog-to-digital (A/D) conversion; and (2) adigital output which is typically available in an industry standardinterface such as an SPI (Serial Peripheral Interface) or I2C(Inter-Integrated Circuit) interface.

As will be appreciated by persons skilled in the art, the magnetometer176 is a sensor which converts magnetic field strength, and in someembodiments the magnetic field strength and direction, into anelectrical signal, producing a corresponding change in output.Magnetometers provide the functionality of a compass. In someembodiments, the magnetometer 176 is a Hall Effect sensor, varyingoutput voltage in response to the magnetic field. Magnetometers mayproduce digital or analog output signals depending on the type ofmagnetometer. The magnetometer may produce digital or analog outputsignals. Generally, two types of outputs are available depending onwhether an analog or digital magnetometer is used: (1) an analog outputrequiring buffering and analog-to-digital (A/D) conversion; and (2) adigital output which is typically available in an industry standardinterface such as an SPI (Serial Peripheral Interface) or I2C(Inter-Integrated Circuit) interface.

Reference is now made to FIG. 5, illustrating a method 500 for detectinga motion based input. A computing device 130 having, memory 120 storingone or more input models 192, a motion-detection subsystem 170 and aprocessor 102 is suitable for performing the method 500. Instructionsfor performing the method 500 may be stored in a non-transitory computerreadable medium as an application, such as the gesture-recognitionapplication 181 stored in memory 120.

In some embodiments, the OS 178 is configured such that processor 102executes gesture-recognition application 181 in the background when thecomputing device 130 is turned on, such that motion-detection subsystem170 is configured to sample data continuously. In some embodiments, thedata samples are obtained at predefined sampling intervals, which may bea regular interval, for example a data sample is obtained every 10 mswhen a user interaction is detected and then every 100 ms when userinteraction is no longer detected.

When a short sampling interval is used, the motion-detection subsystem170 will collect more data, thus requiring more resources, such asmemory, processing power, power consumption, and other resources.Accordingly, it is beneficial to optimize the sampling interval fordifferent scenarios. In some embodiments, the sampling interval isvaried depending on the last known state of the device. For example, thesampling interval may be shortened when the imminent user interaction isexpected, such as when the display 112 is turned on, or when a phonecall is received. However, when user interaction is not expected, suchas when the display 112 is turned off, reducing the sample interval willhelp reduce power consumption of the motion-detection subsystem 170.Other optimizations of the sampling intervals are also possible.

In some embodiments, the sampled data from the motion-detectionsubsystem 170 is then saved in a buffer, for example in RAM 108, and/orin memory 120 as detected-motion 194. The detected-motion 194 representsthe motion detected using the motion-detection subsystem 170, allowingfor processing of the data by the processor 102. The detected-motion 194may be stored in the time-domain by storing a value of the amplitude foreach sensor input provided by the motion-detection subsystem 170 and acorresponding time value for each of one or more sensor inputs. Forexample, the three-axis accelerometer 172 and the three-axis gyroscope174 each provide amplitude measurements of each axis at each samplinginterval, and the magnetometer 176 provides a value of the detectedmagnetic field strength. Accordingly, the sampled data for the varioussensor combinations can be combined in memory with detected-motion 194.Examples of time-domain representations of one of one or more sensorinputs provided by the motion-detection subsystem 170 are shown in FIGS.6, 7A and 7B by plots 604, 702 and 704 respectively. The plots mayrepresent one amplitude value, as measured over time, of one of theamplitude values measured by motion-detection subsystem 170.

The time-domain representation of the detected motion 194 can also beconverted into the frequency domain by the processor 102 applying aFourier transform function, such as a Fast-Fourier transform (FFT). Thefrequency domain data may also be saved in the buffer, for example inRAM 108, and/or in memory 120 as a separate data variable, or combinedin memory with the detected motion 194. By converting to the frequencydomain, the processor 102 is able to process the sampled data moreefficiently in some cases.

In example embodiments, the computing device 130 is configured toperform certain steps upon detecting predetermined motions of thedevice, the motions associated with gesture inputs. In this regard, FIG.5 illustrates a set of steps that the computing device 130 is configuredto perform in example embodiments. With reference to method 500 of FIG.5, at step 502, when the computing device 130 is moved, themotion-detection subsystem 170 detects the motion of the device andprovides sensor data characterizing the movement to the processor 102.The sensor data may be buffered in RAM 108 as detection motion 194. Theprocessor 102 then determines (step 504) if the detected motion 194matches an input model 192 associated with a gesture input. A gestureinput corresponds to a predefined motion that is associated with apredefined device action that the processor 102 is configured toimplement.

In this regard, in some embodiments, one or more input models 192 arestored in the memory 120 of the computing device 130; each input model192 is associated with a gesture input, which in turn is associated witha resulting device action. For example, one input model 192 may definethe action of moving a computing device 130 to one's ear, with theresulting device action being initiation of a phone call session. Theinput model 192 may be stored in the time-domain and include expectedparameter ranges such as values of the amplitude for one or more sensorinputs over a sample duration, corresponding with an expected pattern ofdevice motion; such that the processor 102 is able to match the inputmodel 192 with the detected motion 194 obtained when the motion isdetected. One example of an a time-domain representation of an amplitudevalue of one of one or more sensors inputs as defined in an exampleinput model 192 is shown in FIGS. 6A, 7A and 7B (plot 602). A secondexample of an a time-domain representation of an amplitude value of oneof one or more sensors inputs as defined in an example input model 192is shown in FIG. 6B (plot 606).

Each user of a device 130 may operate the device 130 differently, suchthat the characteristics of the detected motion 194 obtained when themotion corresponding to the gesture input is detected is different foreach user. Accordingly, to reduce a potential bias, in some exampleembodiments the stored input model 192 may be the result of averaging anumber of sample input data.

In some embodiments, the input model 192 is stored in memory 120 at anystage of manufacture or configuration of the computing device 130. Theinput model 192 may be generated by averaging a number sample inputdata, generated by one or more users using one or more computing devices130. In other embodiments, the stored input model 192 is downloaded froma server to memory 120 via communication network 150 during deviceinitialization or during a device update procedure, and may be generatedby averaging a number of sample input data generated by users ofmultiple devices 130 and uploaded to the server. In other embodiments,the stored input model 192 in device 130 is generated or updated locallyby averaging a number of sample input data generated using themotion-detection subsystem 170 of the device 130. In such embodiments,the stored input model 192 may correspond to a model based on a defaultmodel that is adapted based on motion generated by the user or users ofthe subject device 130. Such adapting can occur during a calibrationroutine that could for example be performed during a device setup orinstallation. In some embodiments the input model 192 could be adaptedover time based on usage patterns.

The stored input model 192 may include characteristics associated withone or more sensors 310 of the motion-detection subsystem 170. However,the stored input model 192 may only include accelerometer data, evenwhen the motion-detection subsystem 170 includes both an accelerometer172 and a gyroscope 174.

The input model 192 may define one or more criteria to be consideredwhen determining if a detected motion 194 is associated with the inputmodel 192. In some embodiments, the criteria will include at least oneof: (1) the displacement of the detected motion 194, correlating withthe distance the device 130 is moved when the gesture input isperformed; (2) the speed of the detected motion 194, correlating withhow fast the device 130 is moved when the gesture input is performed;(3) the direction of the displaced distance of the detected motion 194;(4) the frequency response of the detected motion 194, which may beuseful in filtering out unintentional tremors and vibrations in thedetected motion 194; and (5) data from other device components, such asthe proximity sensor 162 or the display 112, which can provideadditional information regarding the detected motion 194 in someembodiments. Each criteria of the input model 192 may be comparedindividually against the detected motion 194. A combination of all thecompared criteria may then be used to determine at step 504 if thedetected motion 194 matches the stored input model 192 and at step 508the confidence level associated with the matching. In some embodiments,a weighting function is applied to each criterion based on thesignificance of the criterion.

At step 504, the processor 102 matches the detected motion 194 with theinput model 192. This is described with reference to FIGS. 6A, 6B, 7Aand 7B, showing graphs 600, 650, 700 and 750 respectively. Each plotillustrates, in the time-domain, an example amplitude of an input fromone sensor of the motion-detection subsystem 170 (plots 604, 604, 702and 704 respectively). FIGS. 6A, 7A and 7B also illustrates, in thetime-domain, a first example amplitude value of a sensor input asdefined in an example input model 192 (plot 602). The example inputmodel shown in plot 602 may correspond to data from one axis of anaccelerometer, showing a steady acceleration with minor variations inthe rate of change of the acceleration over time, as shown by thesteepness of the plot 602. FIG. 6B illustrates, in the time-domain, asecond example amplitude value of a sensor input as defined in anexample input model 192 (plot 606); as in some embodiments, more thanone input model, each representing a different motion, may be stored inthe memory 190 of the computing device 130, and as such, more than oneinput model will be used for matching at step 504.

The sample device motion as illustrated by plot 604, shown in FIG. 6A,may represent data from one axis of the accelerometer 172, showing aninitial acceleration of the device 130 between time 0 ms to time 100 ms,followed by an oscillation of the device 130 between time 200 ms to time500 ms. Accordingly, the example device motion shown in plot 604 doesnot match the characteristics of the example input model shown in plot602. The processor 102 may determine the same by comparing thecharacteristics of the example input model shown in plot 602 with thecharacteristics of the example device motion shown in plot 604,including but not limited to calculating and comparing in real-time thesteepness of each plot, the pattern of acceleration of each plot, thetotal displaced distance corresponding to the each plot, the frequencyresponse of each plot, and other characteristics. In this example, theprocessor 102 may further determine that a match does not exist.

However, the sample device motion as illustrated by plot 604 may bedetermined to match the second example input model shown in plot 606 ofFIG. 6B. The second example input model represents a behavior similar tothat of the sample device motion shown in plot 604. The second exampleinput model may be associated with a different gesture input than thefirst example input model. By storing more than one input model in thememory, the computing device 130 is then able to recognize more than onemotion, each representing a different action. Accordingly, at step 504,the processor 102 determines which input model is associated with thedetected motion, if more than one input model is stored in memory. If noinput models match the detected motion, then the detected motion may beignored.

The sample device motion as illustrated by plot 702, shown in FIG. 7A,may represent data from one axis of the accelerometer 172, showing asteady acceleration with variations in the rate of change of theacceleration over time, as shown by the steepness of the plot 702.Accordingly, sample device motion as illustrated by plot 702 may bedetermined to match the characteristics of example input model shown inplot 602, as previously explained. While variations exist between thesample device motion as illustrated by plot 702 and the example inputmodel shown in plot 602, the characteristics of both motions aresufficiently similar for the processor 102 to determine that the motionassociated with the sample device motion of plot 702 is associated withthe gesture input of the input model shown in plot 602.

If the processor 102 determines that the detected motion 194 matches aninput model 192 associated with a gesture input at step 504, theprocessor 102 then determines an action associated with the gestureinput at step 506. The action may be defined by an instruction or aseries of instructions stored in memory 120, and thus the processor 102only needs to retrieve the instructions from memory 120.

However, in some embodiments, the action is based on a temporal context;i.e. a state of an application when the motion is detected. The OS 178of the computing device 130 allows for multi-tasking, thus gesturerecognition application 181 may run in the background while anotherapplication is run in the foreground. One such application may be thecontacts application 182. Prior to detecting the motion, the processor102 receives a selection indicating selection of a contact entry in thecontact application 182, the contact having a phone number stored inmemory 120. The processor 102 thus determines that the action associatedwith the gesture input, given the temporal context (i.e. the selectionof the contact), is to place a call to the phone number by dialing thenumber using phone application 187. In some embodiments, the contact mayhave a VoIP address stored in memory 120. The processor 102 thusdetermines that the action associated with the gesture input, given thetemporal context (i.e. the selection of the contact having a VoIPaddress), is to place a VoIP call or video call, using VoIP application186, to the VoIP address.

Another such application may be the call log application 183. Prior todetecting the motion, the processor 102 receives a selection indicatingselection of an entry in the call log application 183, the entry havinga phone number stored in memory 120. The processor 102 thus determinesthat the action associated with the gesture input, given the temporalcontext (i.e. the selection of the entry), is to place a call to thephone number by dialing the number.

Another such application may be the email application 184. Prior todetecting the motion, the processor 102 receives a selection of areceived email in the email application, the received email having anemail address associated with it. The processor 102, in someembodiments, determines that the action associated with the gestureinput, given the temporal context (i.e. the selection of the emailmessage), is to send an automated reply message to the sender of thereceived email. This may be beneficial when a user is driving, forexample, and the automated reply message can be configured to indicateso. In other embodiments, the processor 102 detects that a phone numberis embedded in the email message, or that the sender of the emailmessage is associated with a contact entry in the contacts application182, the entry having at least one phone number stored in memory 120.The processor 102 that the action associated with the gesture input,given the temporal context (i.e. the selection of the email messagehaving a phone number associated therewith), is to place a call to thephone number by dialing the number.

Another such application may be the SMS application 185. For example,selection of a received SMS message may trigger actions similar toselection of a received email message. A phone call may be placed to thesender of the message, or an automated reply message may be sent to thesender of the message.

If the processor 102 determines that the detected motion 194 is matchesthe input model at step 504, the processor 102 then determines aconfidence level associated with the match at step 508, representing theconfidence in determining that the detected motion 194 is associatedwith the gesture input. For example, sample device motion of plot 702 ofFIG. 7A more closely represents the sample input model of plot 602 thansample device motion of plot 704 of FIG. 7B. However, the sample devicemotion of plot 704 is indeed associated with a motion represented by thesample input model of plot 602, however, having a faster displacementthan expected by sample input model of plot 602. Accordingly, theconfidence level determined by the processor 102 for the sample devicemotion of plot 702 will be higher than for the sample device motion ofplot 704.

In determining the confidence level, the processor 102 may apply afunction to calculate a difference between a detected motion 194 and thestored input model 192. In one embodiment, each sensor input isconsidered individually, and each criterion defined by the input model,as previously explained, is considered individually for each sensorinput. Accordingly, a plurality of differences may be computed. Forexample, in considering the distance displaced of the detected motion194 in comparison with the distance displaced in the input model 192,the processor 102 may consider this individually from speed criterion.Thus, sample device motion of plot 702 and sample device motion of plot704 are determined by the processor 102 to have the same displaceddistance but at different speeds. The match percentage for thedisplacement parameter is therefore deemed to be the same for bothsamples, whereas the match percentage for the speed parameter istherefore deemed to be lower for sample of plot 704. The sample of plot704 may simply correlate to the same motion occurring at a faster speed,for example, the user was in a rush.

In some embodiments, each of the sensor inputs and each of the criteriaare then given a weighting factor to determine their significance in thematch with the stored input model 192. A default weighting factor isstored in memory 120 during device manufacture and/or configuration.Furthermore, different weighting factors may be applied to differentinput models 192 when more than one input model is stored in memory 120.The processor 102 may compute a value for the confidence level byapplying the weighting factors to a combination of the differences ofall the sensor inputs and criteria to the input model 192. For example,a weighted average value may be computed.

In other embodiments, additional sensor inputs, such as proximity sensor162, are considered as additional criteria in determining the confidencelevel. For example, the processor 102 may enable the proximity sensor162 to sample data at a high sample rate, for example every 100 ms, whenmotion is detected by the motion-detection subsystem 170. The gestureinput may be associated with a certain pattern of proximity data, asstored in the input model 192. For example, a gesture indicatingplacement of a phone call by moving the computing device 130 to a user'sear, the proximity sensor may detect decreasing proximity to an object(i.e. the head and ear).

In some embodiments, a threshold condition may be pre-defined and storedin memory 120, for example to require the value of the confidence levelas determined at step 508 for a detected motion 194 to be above apre-determined threshold, such as an 70 to 80% confidence level. Whenthe confidence level is above the pre-determined threshold, the actionassociated with the gesture input is performed automatically at step518. By requiring a high confidence level threshold, false-positives, indetermining that the detected motion 194 is associated with an inputmodel 192 and an action, are reduced; thus the chances that thecomputing device 130 performs an unwanted action are reduced.

When the confidence level is below the pre-determined threshold, theaction associated with the gesture input is not performed by theprocessor 102 unless confirmation of the gesture input is received atthe processor at step 516, thus ensuring that a gesture detected withlow confidence is still performed, however, only after confirmation toreduce the chances of a false positive.

Optionally, the processor 102 may prompt the user to provideconfirmation of the gesture input at step 514. Furthermore, depending onthe nature of the action, the confirmation may be provided to the userand received by the processor 102 by different methods. For example,when the action requires placement of a phone call, the prompt may be avoice prompt provided to the user using the speaker 158 and theconfirmation may be a voice confirmation received by microphone 158. Inperforming the voice prompt and voice confirmation, thegesture-recognition application 181 may call a function of speechapplication 188, to provide text-to-speech and speech-to-text conversionfunction. The voice prompt may for example ask: “Do you wish to make acall?” to which the voice confirmation may result in a positive “Yes”reply or a negative “No” reply. In some embodiments, for example, when acontact having more than one phone number is selected, the voiceconfirmation may prompt the user to choose a phone number for dialing,and the confirmation may result in receiving a voice input indicating achosen phone number or a voice input indicating that no phone call wasrequested. Accordingly, the prompt may present itself as an opportunityto gain both the confirmation of the action to be performed andadditional input.

In other embodiments, the prompt is displayed on the screen, and theconfirmation is received via an input device 106, such as touchsensitivedisplay 112.

The confirmation received at step 516 may either be negative orpositive. However, in some embodiments, no confirmation is received.This may be due to a false positive; thus the user may not notice that aconfirmation is required. When no confirmation is received, it istherefore considered to be a negative confirmation. When a positiveconfirmation is received, the processor 102 performs the actionassociated with the gesture input at step 518.

In some embodiments, when the confidence level is above thepre-determined threshold, the processor 102 still prompts the user toprovide confirmation of the gesture input (not shown). However, theconfirmation will not be required for all gesture inputs having aconfidence level above the pre-determined threshold. Only a selectedportion of gesture inputs having a confidence level above thepre-determined threshold will be confirmed, for example, one input forevery 20 inputs, or one for every 50 inputs is confirmed. The remaininginputs are not confirmed, thereby requiring the user to perform fewersteps to perform the action. However, when a confirmation is required,the number of false positives can be reduced, for example, as explainedpreviously.

Additionally and optionally, to better adapt and tailor the method 500for future use by the user of the device 130, the processor 102 mayexecute one or more steps of optional steps 520, 524 and 526. Theoptional steps may be executed when the confidence level is above thepre-determined threshold, or the gesture input has been positivelyconfirmed or negatively (not shown) confirmed. For a detected motion 194having a confidence level above the pre-determined threshold theprobability of a false-positive is low in determining that the detectedmotion 194 is associated with an input model 192. Additionally, when adetected motion 194 has been positively or negatively confirmed by theuser, the confidence level associated with the confirmed detected motion194 is considered to have a higher confidence level than previouslydetermined.

At step 520, the detected motion 194 is stored in memory 120 as anadditional input model. Further additional input models may be stored asthe method 500 is repeated. The additional input model or models allowthe processor 102 to compare later detected device motion with both theoriginal input model 192 and the additional input model or models. Whenthe processor 102 then determines if the detected motion 194 matches aninput model 192 associated with a gesture input at step 504 anddetermines the confidence level at step 508, the processor may averagethe values of all the stored input models, applying a weighting factorto each additional input model corresponding to the confidence levelassociated with it. Furthermore, negatively confirmed input models maybe used to identify false-positives. For example, after comparing thedetected motion 194 with all positively confirmed input models, theprocessor 102 may also compare the detected motion 194 to ensure that itdoes match a negatively confirmed input model. As the additional inputmodel is an indicator of a user's past behavior, it may provide a goodindication of the user's future behavior.

In some embodiments, the OS 178 of the computing device 130 isconfigurable for multiple users. Accordingly, each additional inputmodel may be associated with one of the multiple users, and only usedfor comparison purposes for only the user associated with it.

At step 524, the processor 102 may optionally upload the detected motion194 to a server (not shown) via communication network 150. The detectedmotion 194 for uploading may either be positively confirmed vector, ormay be associated with the gesture input with high confidence level. Theserver thus receives detected motions from multiple computing devices130 over time, each associated with the same gesture input. The detectedmotion received at the server can then be analyzed, for example byaveraging, to eliminate outliers and to define an input model associatedwith the gesture input based on a large number of inputs. The inputmodel defined can then be sent to computing devices 130 over time, viacommunication network 150, and stored as input models 192; thusimproving the correlation between the stored models and user behavior.The improved correlation will accordingly in some embodiments helpreduce the number of false-positives and the number of events thatrequire user confirmation.

When a negative or positive confirmation is received, the processor 102may optionally adjust the stored input model 192 at step 526. The storedinput model 192 may be adjusted by varying the weighting factors appliedto the different criteria of the model in correspondence with thereceived confirmation. For example, the criterion associated with thespeed of the detected motion may be identified as a less importantcriterion in matching the detected input with an input model. In oneexample, multiple detected inputs corresponding with the same gestureinput, each performed at a different time and at different speeds; thusthe criterion associated with the speed of the detected motion is givena lower weighting factor. By adjusting the stored input model 192, theinput model becomes more adapted to a user's behavior over time, thusover time allowing for higher confidence levels when determining if thedetected motion is associated with the gesture input and thus requiringfewer confirmations in future.

The steps and/or operations in the flowcharts and drawings describedherein are for purposes of example only. There may be many variations tothese steps and/or operations without departing from the teachings ofthe present disclosure. For instance, the steps may be performed in adiffering order, or steps may be added, deleted, or modified.

While the present disclosure is described, at least in part, in terms ofmethods, a person of ordinary skill in the art will understand that thepresent disclosure is also directed to the various components forperforming at least some of the aspects and features of the describedmethods, be it by way of hardware components, software or anycombination of the two, or in any other manner. Moreover, the presentdisclosure is also directed to a pre-recorded storage device or othersimilar computer readable medium including program instructions storedthereon for performing the methods described herein.

The present disclosure may be embodied in other specific forms withoutdeparting from the subject matter of the claims. The described exampleembodiments are to be considered in all respects as being onlyillustrative and not restrictive. The present disclosure intends tocover and embrace all suitable changes in technology. The scope of thepresent disclosure is, therefore, described by the appended claimsrather than by the foregoing description. The scope of the claims shouldnot be limited by the described embodiments set forth in the examples,but should be given the broadest interpretation consistent with thedescription as a whole.

1. A method implemented by a computing device having a memory and amotion detection sensor, the memory storing an input model associatedwith a gesture input, the method comprising: detecting a motion of thecomputing device; matching the detected motion with the input model;determining a confidence level associated with the match; if theconfidence level is above a pre-determined threshold, performing anaction associated with the gesture input; and if the confidence level isbelow the pre-determined threshold, performing the action responsive toreceiving a positive confirmation of the gesture input.
 2. The method ofclaim 1, further comprising determining the action associated with thegesture input based on a temporal context.
 3. The method of claim 2,wherein the temporal context is a state of an application when themotion is detected.
 4. The method of claim 1, wherein determining theconfidence level comprises comparing the detected motion with the inputmodel.
 5. The method of claim 1, further comprising storing the detectedmotion in the memory as an additional input model in response toreceiving the positive confirmation or a negative confirmation of thegesture input.
 6. The method of claim 1, further comprising adjustingthe input model in response to receiving the positive confirmation or anegative confirmation of the gesture input.
 7. The method of claim 1,further comprising: prior to detecting the motion, receiving a selectionindicating selection of a message; and wherein performing the actioncomprises sending a reply to the message.
 8. The method of claim 1,further comprising: prior to detecting the motion, receiving a selectionindicating selection of a contact entry, the contact entry having aplurality of phone numbers stored in memory; and wherein performing theaction comprises: prompting a user to select a phone number of theplurality of phone numbers for dialing; receiving a voice inputselecting the phone number; and dialing the phone number.
 9. The methodof claim 1, wherein receiving the positive confirmation comprises:prompting a user to confirm the gesture input; and receiving an inputindicating the positive confirmation.
 10. The method of claim 9, whereinprompting the user comprises prompting the user via a voice prompt andreceiving a voice input indicating the positive confirmation.
 11. Acomputing device comprising: memory for storing instructions and forstoring an input model associated with a gesture input; a motiondetection sensor; a processor coupled to the memory and the motiondetection sensor, wherein the processor is configured to: detect amotion of the computing device; match the detected motion with the inputmodel; determine a confidence level associated with the match; if theconfidence level is above a pre-determined threshold, perform an actionassociated with the gesture input; and if the confidence level is belowa pre-determined threshold, perform an action responsive to receiving apositive confirmation of the gesture input.
 12. The computing device ofclaim 11, wherein the processor is further configured to determine theaction associated with the detected motion based on a temporal context.13. The computing device of claim 12, wherein the temporal context is astate of an application when the motion is detected.
 14. The computingdevice of claim 11, wherein the processor is further configured tocompare the detected motion with the input model stored in memory whendetermining the confidence level.
 15. The computing device of claim 11,wherein the processor is further configured to store the detected motionin memory as an additional input model in response to receiving thepositive confirmation or a negative confirmation of the gesture input.16. The computing device of claim 11, wherein the processor is furtherconfigured to adjust the input model in response to receiving thepositive confirmation or a negative confirmation of the gesture input.17. The computing device of claim 11, wherein the processor is furtherconfigured to receive, prior to detecting the motion, a selectionindication selection of a message, and wherein performing the actioncomprises sending a reply to the message.
 18. The computing device ofclaim 11, wherein the processor is further configured to receive, priorto detecting the motion, a selection indicating selection of a contactentry, the contact entry having a plurality of phone numbers stored inmemory; and when performing the action: prompt a user to select a phonenumber of the plurality of phone numbers for dialing; receive a voiceinput selecting the phone number; and dial the phone number.
 19. Thecomputing device of claim 11, wherein the processor is furtherconfigured to prompt a user to confirm the gesture input, and receive aninput indicating the positive confirmation.
 20. The computing device ofclaim 19, wherein the processor is further configured to prompt the uservia a voice prompt and receive a voice input indicating the positiveconfirmation.